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8.5.5 Solved Problems 


Problem 1 

Consider the following observed values of (x p v ; ): 

(-1,6), (0,3), (1,2), (2,-1) 

a. Find the estimated regression line 

A A 

y = A 0 +Ai*, 

based on the observed data. 

b. For each x p compute the fitted value of v, using 


A A 

y i = A 0 + h x i- 


c. Compute the residuals, e i = y i - v ; . 

d. Find /?-squared (the coefficient of determination). 

• Solution 

o a. We have 


- 1 + 0 + 1+2 


6 + 3 + 2 + (-l) 
v = - 4 - = 2-5, 

s xx = ( - 1 - 0.5) 2 + (0 - 0.5) 2 + (1 - 0.5) 2 + (2 - 0.5) 2 = 5, 
■s' xv = ( - 1 - 0.5)(6 - 2.5) + (0 - 0.5)(3 - 2.5) 

+ (1 - 0.5)(2 - 2.5) + (2 - 0.5)( - 1 - 2.5) = - 11. 

Therefore, we obtain 


At 



- 2 . 2 , 


J3 Q = 2.5 - ( - 2.2)(0.5) = 3.6 
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The following MATLAB code can be used to obtain the estimated 
regression line 
x=[-l;0;l;2]; 
xO=ones(size(x)); 
y=[6;3;2;-l]; 
beta = regress(y,[xO,x]); 

b. The fitted values are given by 


yl = 3.6 - 2.2 Xj, 


so we obtain 


y\ = 5.8, h = 3 - 6 ’ h = L4 ’ y4 = ~ °- 8 

c. We have 

<?l = Vj ~y \ = 6 - 5.8 = 0.2, 

^2 = T2 ~Yl = 3 - 3 - 6 = " °- 6 ’ 
e 3 = T 3 -y 3 = 2 - 1.4 = 0.6, 

e 4 = y4~y4 = - i -(-o. 8 ) = -0.2 

d. We have 

s = (6 - 2.5) 2 + (3 - 2.5) 2 + (2 - 2.5) 2 + ( - 1 - 2.5) 2 = 25. 
We conclude 


(~11) 2 
5 x 25 


0.968 


Problem 2 

Consider the model 


Y=p o+frX+e, 


where c is a N( 0, a 2 ) random variable independent of X. Let also 


Y = P 0 + PiX. 


Show that 


E[(Y~EY) 2 ] = E[(Y- EY) 2 ] + E[(Y- Y) 2 ]. 
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• Solution 

o Since X and c are independent, we can write 

Var( 7) = /i,Var(A) + Var(r) (8.10) 


Note that, 


Y-EY={fi 0 +p l A(]-{P q +P x EX) 
= P x (X~EX). 


Therefore, 


E[(Y-EY) 2 ] = p jVar(Z). 


Also, 

E[( Y~ EY ) 2 ] = Var(7), £[(7- 7) 2 ] = Var(e). 

Combining with Eq uation 8.10 . we conclude 

E[(Y~EY) 2 ] = £[(7-£7) 2 ] +£[(7- 7) 2 ]- 


Problem 3 


A A 

Show that, in a simple linear regression, the estimated coefficients p {] and p | (least squares 
estimates of fl () and f> \ ) satisfy the following equations 


n 


T e < = °’ 

i= 1 


n 


H e t x i = °> 

i= 1 


n a 


Z e rT/ = 0, 

i= 1 


A A A A A 

where e i = y t - v, = y t ~ (> {) ~ fi x x. Hint: /i () and /l, satisfy Eq uation 8.8 and Eq uation 8.9 . By 
cancelling the ( - 2) factor, you can write 


« A A 

Z(7~A)“^i x /) = °> 

i= 1 

n a a 

Z(f/ _ Ao“^i x />/ = 0 - 

/= l 
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Use the above equations to show the desired equations. 

• Solution 

o We have 


n a a 

Z 0/ - A) ~ P\ x i) = 

i= 1 

n A A 

Z (u \ x i) x i = °- 

i= 1 

A A 

Since e f = ~ J3q ~ fi\X, we conclude 

n 

Z e i = 0 ’ 

i= 1 
n 

Z^ = °- 

i= 1 

Moreover, 

w A n A A 

Z e rU = Z e //4 + / i l x /) 

/= 1 /= I 

a n a n 

= AoZ^ + ^iZ^ 

/= i /= i 

= 0 + 0 = 0 . 


Problem 4 

Show that the coefficient of detennination can also be obtained as 

A - 

l”=tCv/-v ) 2 

r 2 = --—. 

Z ?= i ( y ,-^) 2 

• Solution 

o We know 
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y-i = Po + P\ x ii 


y = A 0 + Ai*- 


Therefore, 


Therefore, 


n a - « 


I (*->» 2 = Z^ix-Aix ) 2 

/ = 1 /= I 


AiZ(*/ - *) 2 

/= 1 

f j2 \ s xx- 


A - 

l”=iCv/-v ) 2 


V« , x2 xr 

Z/=i(T/-y) 

5 xy 5 jcy 

= - (since Ai = —) 

s xx s vy s xx 


Problem 5 

(The Method of Maximum Likelihood) This problem assumes that you are familiar with the 
maximum likelihood method discussed in Section 8.2.3 . Consider the model 

Y i = f j 0 + f j \ x i + e P 

where e-s are independent N( 0, a 2 ) random variables. Our goal is to estimate f> () and A| • We 
have the observed data pairs (x 1? y j), (x 2 ,y 2 ), ( x n ,y n ). 

a. Argue that, for given values of J3 0 , Ai, and x ; -, Y i is a nonnal random variable with mean 
A 0 + A| x , an d variance u 2 . Moreover, show that the Y-s are independent. 

b. Find the likelihood function 
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L(y\,y2> f y{y 2 ---Y^y i , ^ ; 2’ Ao’Ai)- 

c. Show that the maximum likelihood estimates of/? 0 and /i| are the same as the ones we 
obtained using the least squares method. 

• Solution 

o a. Given values of /3 0 , /?j, and x-, c = /1 0 + y 5 1 x / is a constant. Therefore, 

= c + r ( is a normal random variable with mean c and variance a . Also, 
since the <r/s are independent, we conclude that Y- s are also independent 
random variables. 

b. By the previous part, for given values of Pq,P\, and x t , 

fyiv-JoJi) = /- 2 e x P (- r(y - Ao “ P\ x i ) 2 

■\j27va 2 ( ^ 

Therefore, the likelihood function is given by 

1 ’ 72’ '"’Tr I’fio’fil) = '"’yn’Po>Pd 

= fyjy i i P i )/r 2 (y2; fio’Pi)" 'fyjy^ Ao’ P i) 

! n 

-oZo-Ao-Ai^) 2 

r= 1 

c. To find the maximum likelihood estimates (MLE) of/? 0 and ft |, we need to 

A A 

find /? 0 and /?! such that the likelihood function 

! 1 ^ 

-~X(y-A 0 -/W 2 

/= i 

is maximized. This is equivalent to minimizing 

n 

Z (y-Ao-M) 2 - 

i= 1 

The above expression is the sum of the squared errors, gift 0 , fly) (Eq uation 
8/7 ). Therefore, the maximum likelihood estimation for this model is the 
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same as the least squares method. 


<— previous 
next —■» 
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